Apparatus, system and method for detecting onset Autism Spectrum Disorder via a portable device

ABSTRACT

Technologies and techniques for detecting and alerting onset Autism Spectrum Disorder (ASD), where a device receives sensor data including heart rate data, accelerometer data and audio data. The device is configured to activate the collection of accelerometer data and extract accelerometer data features comprising frequency components if the heart rate exceeds a first threshold, activate the collection of audio data and extract audio data features comprising frequency components if the heart rate data meets or exceeds a second predetermined threshold, determine if the audio data meets or exceeds a predetermined audio threshold comprising frequency characteristics, and transmit an alert indicating the detection of an onset ASD episode, based on the extracted audio data features, the extracted accelerometer data features and the heart rate data.

FIELD OF TECHNOLOGY

The present disclosure is directed to technologies and techniques fordetecting onset Autism Spectrum Disorder via a portable device. Morespecifically, the present disclosure is directed to detecting usercharacteristics in a tiered configuration to improve detection of onsetAutism Spectrum Disorder, while improving efficiency of the portabledevice during the course of the detection.

BACKGROUND

Autism Spectrum Disorder (ASD) is a neurodevelopmental disordercharacterized by persistent difficulties in social communication, socialinteraction, and restricted, repetitive patterns of behavior, interests,or activities that are present in the early developmental period. ASDrefers to a spectrum of disorders with a range of manifestations thatcan occur on different degrees and in a variety of forms. To address themany challenges faced by children and adults with ASD, the role ofsensing technologies has increased in recent years. These technologiesare generally aimed at providing assistance in overcoming theselimitations, allowing such individuals to understand and participate inthe world around them. Because ASD is not a neurodegenerative disorder,many of the core symptoms can improve as the individuals learn to copewith their environments under the right conditions. The earlier the ageat which intervention can be started, the better their learning anddaily function can be facilitated. Hence, sensing technologies can playa key role in the screening and therapy of ASD, thus potentiallyimproving the lives of those on the spectrum.

Of the existing sensor-based systems, many provide certain benefits fordetecting ASD in users. However, many of these systems utilize expensivesensors and/or equipment that are not practical for home use. Somesystems utilize smart phones as a means to process sensor data to detectASD. However, existing smart phone based ASD systems can be inaccurateand are not computationally efficient, resulting in slower smart phonefunction during sensing operations. Additionally, the inefficientutilization of phone sensors (e.g., camera, fingerprint scanner,microphone) often causes excessive drain on the smart phone battery,which may result in periods of inoperability for the phone. What isneeded are technologies and techniques for accurately detecting ASDwhile increasing efficiency for a portable monitoring device.

SUMMARY

Various apparatus, systems and methods are disclosed herein relating todetecting onset ASD in a tiered manner.

In some illustrative embodiments, a system is disclosed, for detectingand alerting onset Autism Spectrum Disorder (ASD), comprising aprocessing apparatus; a memory operatively coupled to the processingapparatus; a communication interface, operatively coupled to theprocessing apparatus; and an input/output interface, configured toreceive sensor data comprising heart rate data, accelerometer data andaudio data, wherein the processing apparatus is configured to determineif the heart rate data meets or exceeds a first predetermined threshold,activate the collection of accelerometer data and extract accelerometerdata features if the heart rate exceeds the first threshold, determineif the heart rate data meets or exceeds a second predeterminedthreshold, activate the collection of audio data and extract audio datafeatures if the heart rate data meets or exceeds the secondpredetermined threshold, determine if the audio data meets or exceeds apredetermined audio threshold, and transmit an alert indicating thedetection of an onset ASD episode, based on the extracted audio datafeatures, the extracted accelerometer data features and the heart ratedata.

In some illustrative embodiments, a processor-based method is disclosedfor detecting and alerting onset Autism Spectrum Disorder (ASD) in adevice, comprising: receiving, at an input/output interface, sensor datacomprising heart rate data; determining, in a processing apparatus, ifthe heart rate data meets or exceeds a first predetermined threshold;activating, via the processing apparatus, the collection ofaccelerometer data and extracting accelerometer data features if theheart rate exceeds the first threshold, determining, via the processingapparatus, if the heart rate data meets or exceeds a secondpredetermined threshold, activating, via the processing apparatus, thecollection of audio data and extract audio data features if the heartrate data meets or exceeds the second predetermined threshold,determining, via the processing apparatus, if the audio data meets orexceeds a predetermined audio threshold, and transmitting, via acommunications interface coupled to the processing apparatus, an alertindicating the detection of an onset ASD episode, based on the extractedaudio data features, the extracted accelerometer data features and theheart rate data.

In some illustrative embodiments, a system is disclosed for detectingand alerting onset Autism Spectrum Disorder (ASD), comprising: aprocessing apparatus; a memory operatively coupled to the processingapparatus; a communication interface, operatively coupled to theprocessing apparatus; and an input/output interface, configured toreceive sensor data comprising heart rate data, accelerometer data andaudio data, wherein the processing apparatus is configured to activatethe collection of accelerometer data and extract accelerometer datafeatures comprising frequency components if the heart rate exceeds afirst threshold, activate the collection of audio data and extract audiodata features comprising frequency components if the heart rate datameets or exceeds a second predetermined threshold, determine if theaudio data meets or exceeds a predetermined audio threshold comprisingfrequency characteristics, and transmit an alert indicating thedetection of an onset ASD episode, based on the extracted audio datafeatures, the extracted accelerometer data features, and the heart ratedata.

BRIEF DESCRIPTION OF THE FIGURES

The present invention is illustrated by way of example and notlimitation in the figures of the accompanying drawings, in which likereferences indicate similar elements and in which:

FIG. 1 illustrates a systematic overview of a wireless ASD monitoringsystem including sensors, a portable device, and a processing apparatuscommunicatively coupled to a network under an illustrative embodiment;

FIG. 2 schematically illustrates an operating environment for a portabledevice and a server communicatively coupled to a network for processingASD data under an illustrative embodiment;

FIG. 3 schematically illustrates an operating environment for a portabledevice configured to process ASD data under an illustrative embodiment;

FIG. 4A shows a simplified example of a plurality of sensors wirelesslycommunicating with a portable device under an illustrative embodiment

FIG. 4B shows another simplified example of a portable device utilizingphone sensors under an illustrative embodiment;

FIG. 5 shows an operating environment for processing sensor data frommultiple sources in a portable device and providing a detection value,along with learning, classification and labeling algorithms under anillustrative embodiment;

FIG. 6 shows a simplified process for collecting and processing heartrate data, accelerometer data, and audio data in a portable device todetect and alert for a state of ASD under an illustrative embodiment;

FIG. 7 shows a process for extracting feature vectors for accelerometerdata based on hear rate conditions, and determining audiocharacteristics for detecting and alerting for a state of ASD under anillustrative embodiment; and

FIG. 8 shows a process for collecting and processing global positioningsystem (GPS) data for loading thresholds for use in sensor dataprocessing under an illustrative embodiment.

DETAILED DESCRIPTION

Various embodiments will be described herein below with reference to theaccompanying drawings. In the following description, well-knownfunctions or constructions are not described in detail since they mayobscure the invention in unnecessary detail.

It will be understood that the structural and algorithmic embodiments asused herein does not limit the functionality to particular structures oralgorithms, but may include any number of software and/or hardwarecomponents. In general, a computer program product in accordance withone embodiment comprises a tangible computer usable medium (e.g., harddrive, standard RAM, an optical disc, a USB drive, or the like) havingcomputer-readable program code embodied therein, wherein thecomputer-readable program code is adapted to be executed by a processor(working in connection with an operating system) to implement one ormore functions and methods as described below. In this regard, theprogram code may be implemented in any desired language, and may beimplemented as machine code, assembly code, byte code, interpretablesource code or the like (e.g., via C, C++, C#, Java, Actionscript,Swift, Objective-C, Javascript, CSS, XML, etc.). Furthermore, the term“information” as used herein is to be understood as meaning digitalinformation and/or digital data, and that the term “information” and“data” are to be interpreted as synonymous.

Turning to FIG. 1, a system 100 is shown for collecting and processingdata from one or more sensors (104-116) communicatively coupled to thebody of a user 102. In this example, sensor 108 may be an accelerometersensor, which is known in the art. Generally speaking, an accelerometeris a device that measures the acceleration (or rate of change ofvelocity) of a body in its own instantaneous rest frame. Sensor 108 maybe configured using a single- and/or multi-axis models of accelerometerto detect magnitude and direction of the proper acceleration, as avector quantity, and can be used to sense orientation (because directionof weight changes), coordinate acceleration, vibration, shock, andfalling in a resistive medium (a case where the proper accelerationchanges, since it starts at zero, then increases). In an illustrativeembodiment, sensor 108 may be configured as a micromachinedmicroelectromechanical systems (MEMS) accelerometer, configured withsuitable memory, processing, and a transmitter to process and transmitraw sensor data.

Sensor 106 may be configured as a heart rate monitor, which is known inthe art. Sensor 106 may comprise a transmitter, worn on a chest strap,wrist strap, ankle strap, or any other suitable configuration, and areceiver, where, when a heartbeat is detected, a radio signal istransmitted, which the sensor receiver uses to determine a current heartrate. This signal can be a simple radio pulse or a unique coded signalfrom the chest strap (such as Bluetooth, ANT, or other low-power radiolink); the latter prevents one user's receiver from using signals fromother nearby transmitters (cross-talk interference). Alternately, sensor106 may be configured to use optics to measure heart rate by measuringchanges in blood flow by shining a light from an LED in the sensorthrough the skin and measuring how it scatters off blood vessels. Inaddition to measuring the heart rate, devices using this technology areable to measure blood oxygen saturation (SpO2). Sensor 106 may alsoinclude a microprocessor and suitable memory, along with atransmitter/receiver to simultaneously monitor heart rate, oxygensaturation, and other parameters. These may include sensors such asaccelerometers, gyroscopes, and GPS to detect speed, location anddistance.

Sensor 104 may comprise an audio sensor, which is generally known in theart. Sensor 104 may be configured as a microphone apparatus, configuredto detect atmospheric vibrations that are perceived as sound. In anillustrative embodiment, sensor 104 is configured to detect frequenciesin a broader range than possible by the human ear. Also, sensor 104 maybe configured with a microprocessor and suitable memory, along with atransmitter receiver to process and transmit the raw audio data. In someillustrative embodiments, sensors 104-108 may be separate sensors, eachwith their own processing and transmitting apparatuses. In someillustrative embodiments, sensors 104-108 may be combined with eachother in part, or combined together as one device. Sensors 104-108 maybe configured to be physically worn on the body, or may be incorporatedas peripheral devices on portable device 112, which may be a smartphone, tablet, or any suitable processing device known in the art. Eachof sensors 104-108 are configured to transmit and receive data viawireless node 110 using a suitable wireless communication protocol. Node110 is further configured to communicate data with portable device 112and/or processing device 114. Portable device 112 may also communicatewith processing device 114, and together they may individuallycommunicate over computer network 116.

FIG. 2 shows an operating environment for system 200 that includes aprocessing device 202 (e.g., portable device 112 of FIG. 1) and a server220 communicating via the network 116, wherein the system is configuredto process sensor data under an illustrative embodiment. In theillustrative embodiment, the processing device 202 includes a processor210 or processor circuit, one or more peripheral devices 204,memory/data storage 206, communication circuity 212, and a sensormanager 214. Sensor manager 214 may be configured to process and/ormanage sensor data received from any of the sensors described above inconnection with FIG. 1. The sensor manager 214 may be incorporated intomemory/data storage 206 with or without a secure memory area, or may bea dedicated component, or incorporated into the processor 210. Ofcourse, processing device 202 may include other or additionalcomponents, such as those commonly found in a digital apparatus and/orcomputer (e.g., sensors, various input/output devices), in otherembodiments. Additionally, in some embodiments, one or more of theillustrative components may be incorporated in, or otherwise form aportion of, another component. For example, the memory/data storage 206,or portions thereof, may be incorporated in the processor 210 in someembodiments.

The processor 210 may be embodied as any type of processor currentlyknown or developed in the future and capable of performing the functionsdescribed herein. For example, the processor 210 may be embodied as asingle or multi-core processor(s), digital signal processor,microcontroller, or other processor or processing/controlling circuit.Similarly, memory/data storage 206 may be embodied as any type ofvolatile or non-volatile memory or data storage currently known ordeveloped in the future and capable of performing the functionsdescribed herein. In operation, memory/data storage 206 may storevarious data and software used during operation of the processing device210 such as access permissions, access parameter data, operatingsystems, applications, programs, libraries, and drivers.

Memory/data storage 206 may be communicatively coupled to the processor210 via an I/O subsystem 208, which may be embodied as circuitry and/orcomponents to facilitate input/output operations with the processor 210,memory/data storage 206, and other components of the processing device202. For example, the I/O subsystem 208 may be embodied as, or otherwiseinclude, memory controller hubs, input/output control hubs, firmwaredevices, communication links (i.e., point-to-point links, bus links,wires, cables, light guides, printed circuit board traces, etc.) and/orother components and subsystems to facilitate the input/outputoperations. In some embodiments, the I/O subsystem 208 may form aportion of a system-on-a-chip (SoC) and be incorporated, along with theprocessor 210, memory/data storage 206, and other components of theprocessing device 202, on a single integrated circuit chip.

The processing device 202 includes communication circuitry 212(communication interface) that may include any number of devices andcircuitry for enabling communications between processing device 202 andone or more other external electronic devices and/or systems. Similarly,peripheral devices 204 may include any number of additional input/outputdevices, interface devices, and/or other peripheral devices. Theperipheral devices 204 may also include a display, along with associatedgraphics circuitry and, in some embodiments, may further include akeyboard, a mouse, audio processing circuitry (including, e.g.,amplification circuitry and one or more speakers), and/or otherinput/output devices, interface devices, and/or peripheral devices.

The server 220 may be embodied as any type of server (e.g., a webserver, etc.) or similar computing device capable of performing thefunctions described herein. In the illustrative embodiment of FIG. 2 theserver 220 includes a processor 228, an I/O subsystem 226, a memory/datastorage 224, communication circuitry 232, and one or more peripheraldevices 222. Components of the server 220 may be similar to thecorresponding components of the processing device 202, the descriptionof which is applicable to the corresponding components of server 220 andis not repeated herein for the purposes of brevity.

The communication circuitry 232 of the server 220 may include any numberof devices and circuitry for enabling communications between the server220 and the processing device 202. In some embodiments, the server 220may also include one or more peripheral devices 222. Such peripheraldevices 222 may include any number of additional input/output devices,interface devices, and/or other peripheral devices commonly associatedwith a server or computing device. The server 220 also includes a sensordata manager 230 that is configured to process sensor data from sensormanager 214. The processing from sensor data manager 230 may beconfigured as back-end processing, and/or may be configured to furtherprocess sensor data processed and/or pre-processed in sensor manager214.

In the illustrated embodiment, communication between the server 220 andthe processing device 202 takes place via a network 116 that may beoperatively coupled to one or more network switches (not shown). In oneembodiment, the network 116 may represent a wired and/or wirelessnetwork and may be or include, for example, a local area network (LAN),personal area network (PAN), storage area network (SAN), backbonenetwork, global area network (GAN), wide area network (WAN), orcollection of any such computer networks such as an intranet, extranetor the Internet (i.e., a global system of interconnected network uponwhich various applications or service run including, for example, theWorld Wide Web). Generally, the communication circuitry of processingdevice 202 and the communication circuitry 232 of the server 220 may beconfigured to use any one or more, or combination, of communicationprotocols to communicate with each other such as, for example, a wirednetwork communication protocol (e.g., TCP/IP), a wireless networkcommunication protocol (e.g., Wi-Fi, WiMAX), a cellular communicationprotocol (e.g., Wideband Code Division Multiple Access (W-CDMA)), and/orother communication protocols. As such, the network 116 may include anynumber of additional devices, such as additional computers, routers, andswitches, to facilitate communications between the processing device 202and the server 220.

FIG. 3 is an exemplary embodiment of a portable computing device 300(such as devices 202, 112), and may be a smart phone, tablet computer,laptop or the like. Device 300 may include a central processing unit(CPU) 301 (which may include one or more computer readable storagemediums), a memory controller 302, one or more processors 303, aperipherals interface 304, RF circuitry 305, audio circuitry 306,accelerometer 307, speaker 321, microphone 322, and input/output (I/O)subsystem 221 having display controller 318, control circuitry for oneor more sensors 319 and input device control 320. These components maycommunicate over one or more communication buses or signal lines indevice 300. It should be appreciated that device 300 is only one exampleof a portable multifunction device, and that device 300 may have more orfewer components than shown, may combine two or more components, or amay have a different configuration or arrangement of the components. Thevarious components shown in FIG. 3 may be implemented in hardware or acombination of hardware and software, including one or more signalprocessing and/or application specific integrated circuits.

Memory (or storage) 308 may include high-speed random access memory(RAM) and may also include non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Access to memory 308 by othercomponents of the device 300, such as processor 303, and peripheralsinterface 304, may be controlled by the memory controller 302.Peripherals interface 304 couples the input and output peripherals ofthe device to the processor 303 and memory 308. The one or moreprocessors 303 run or execute various software programs and/or sets ofinstructions stored in memory 308 to perform various functions for thedevice 300 and to process data. In some embodiments, the peripheralsinterface 304, processor(s) 303, decoder 313 and memory controller 302may be implemented on a single chip, such as a chip 301. In otherembodiments, they may be implemented on separate chips.

RF (radio frequency) circuitry 305 receives and sends RF signals, alsoknown as electromagnetic signals. The RF circuitry 305 convertselectrical signals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. The RF circuitry 305 may include well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 305 may communicate with networks, such as the Internet, alsoreferred to as the World Wide Web (WWW), an intranet and/or a wirelessnetwork, such as a cellular telephone network, a wireless local areanetwork (LAN) and/or a metropolitan area network (MAN), and otherdevices by wireless communication. The wireless communication may useany of a plurality of communications standards, protocols andtechnologies, including but not limited to Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), high-speeddownlink packet access (HSDPA), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over InternetProtocol (VoIP), Wi-MAX, a protocol for email (e.g., Internet messageaccess protocol (IMAP) and/or post office protocol (POP)), instantmessaging (e.g., extensible messaging and presence protocol (XMPP),Session Initiation Protocol for Instant Messaging and PresenceLeveraging Extensions (SIMPLE), and/or Instant Messaging and PresenceService (IMPS)), and/or Short Message Service (SMS)), or any othersuitable communication protocol, including communication protocols notyet developed as of the filing date of this document.

Audio circuitry 306, speaker 321, and microphone 322 provide an audiointerface between a user and the device 300. Audio circuitry 306 mayreceive audio data from the peripherals interface 304, converts theaudio data to an electrical signal, and transmits the electrical signalto speaker 321. The speaker 321 converts the electrical signal tohuman-audible sound waves. Audio circuitry 306 also receives electricalsignals converted by the microphone 321 from sound waves, which mayinclude utterances from a speaker. The audio circuitry 306 converts theelectrical signal to audio data and transmits the audio data to theperipherals interface 304 for processing. Audio data may be retrievedfrom and/or transmitted to memory 308 and/or the RF circuitry 305 byperipherals interface 304. In some embodiments, audio circuitry 306 alsoincludes a headset jack for providing an interface between the audiocircuitry 306 and removable audio input/output peripherals, such asoutput-only headphones or a headset with both output (e.g., a headphonefor one or both ears) and input (e.g., a microphone).

I/O subsystem 221 couples input/output peripherals on the device 300,such as touch screen 315, sensors 316 and other input/control devices317, to the peripherals interface 304. The I/O subsystem 221 may includea display controller 318, sensor controllers 319, and one or more inputcontrollers 320 for other input or control devices. The one or moreinput controllers 320 receive/send electrical signals from/to otherinput or control devices 317. The other input/control devices 317 mayinclude physical buttons (e.g., push buttons, rocker buttons, etc.),dials, slider switches, joysticks, click wheels, and so forth. In somealternate embodiments, input controller(s) 320 may be coupled to any (ornone) of the following: a keyboard, infrared port, USB port, and apointer device such as a mouse, an up/down button for volume control ofthe speaker 321 and/or the microphone 322. Touch screen 315 may also beused to implement virtual or soft buttons and one or more softkeyboards.

Touch screen 315 provides an input interface and an output interfacebetween the device and a user. Display controller 318 receives and/orsends electrical signals from/to the touch screen 315. Touch screen 315displays visual output to the user. The visual output may includegraphics, text, icons, video, and any combination thereof. In someembodiments, some or all of the visual output may correspond touser-interface objects. Touch screen 315 has a touch-sensitive surface,sensor or set of sensors that accepts input from the user based onhaptic and/or tactile contact. Touch screen 315 and display controller318 (along with any associated modules and/or sets of instructions inmemory 308) detect contact (and any movement or breaking of the contact)on the touch screen 315 and converts the detected contact intointeraction with user-interface objects (e.g., one or more soft keys,icons, web pages or images) that are displayed on the touch screen. Inan exemplary embodiment, a point orf contact between a touch screen 315and the user corresponds to a finger of the user. Touch screen 215 mayuse LCD (liquid crystal display) technology, or LPD (light emittingpolymer display) technology, although other display technologies may beused in other embodiments. Touch screen 315 and display controller 318may detect contact and any movement or breaking thereof using any of aplurality of touch sensing technologies now known or later developed,including but not limited to capacitive, resistive, infrared, andsurface acoustic wave technologies, as well as other proximity sensorarrays or other elements for determining one or more points of contactwith a touch screen 315.

Device 300 may also include one or more sensors 316 such as heart ratesensors, touch sensors, optical sensors that comprise charge-coupleddevice (CCD) or complementary metal-oxide semiconductor (CMOS)phototransistors. The optical sensor may capture still images or video,where the sensor is operated in conjunction with touch screen display315. Device 300 may also include one or more accelerometers 307, whichmay be operatively coupled to peripherals interface 304. Alternately,the accelerometer 307 may be coupled to an input controller 320 in theI/O subsystem 221. The accelerometer is preferably configured to outputaccelerometer data in the x, y, and z axes.

In some illustrative embodiments, the software components stored inmemory 308 may include an operating system 309, a communication module310, a text/graphics module 311, a Global Positioning System (GPS)module 312, decoder 313 and applications 314. Operating system 309(e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embeddedoperating system such as VxWorks) includes various software componentsand/or drivers for controlling and managing general system tasks (e.g.,memory management, storage device control, power management, etc.) andfacilitates communication between various hardware and softwarecomponents. Communication module 310 facilitates communication withother devices over one or more external ports and also includes varioussoftware components for handling data received by the RF circuitry 305.An external port (e.g., Universal Serial Bus (USB), Firewire, etc.) maybe provided and adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.).

Text/graphics module 311 includes various known software components forrendering and displaying graphics on the touch screen 315, includingcomponents for changing the intensity of graphics that are displayed. Asused herein, the term “graphics” includes any object that can bedisplayed to a user, including without limitation text, web pages, icons(such as user-interface objects including soft keys), digital images,videos, animations and the like. Additionally, soft keyboards may beprovided for entering text in various applications requiring text input.GPS module 312 determines the location of the device and provides thisinformation for use in various applications. Applications 314 mayinclude various modules, including health monitoring software, sensorsoftware, navigation software, mapping, address books/contact list,email, instant messaging, and the like.

FIG. 4A shows a simplified example of a system arrangement 400, whereina first sensor 404 and second sensor 406 communicate wirelessly with aportable device 402 under an illustrative embodiment. As explainedherein, it is understood by those skilled in the art that the specificsensors named herein are merely for exemplary purposes, and that othersensors may be incorporated into the present disclosure. For example,perspiration sensors, blood glucose sensors, eye movement sensors andother suitable sensors are contemplated in the present disclosure. FIG.4B shows another simplified example of a system arrangement 410 whereinthe portable device 402 utilizes phone sensors under an illustrativeembodiment. Here, a portable device 402 camera (not shown) may beutilized by a user to place a finger of a hand 408 over the camera lensto determine heart rate 412.

FIG. 5 shows an operating environment 500 for processing sensor datafrom multiple sources in a portable device and providing a detectionvalue, along with learning, classification and labeling algorithms underan illustrative embodiment. In this example, the operating environmentis broken down into processing blocks 502-506 that may be configured tooperate on a portable device (e.g., 202) and processing blocks 530-534that may be configured to operate on a sensing device (e.g., 104-108).The processing blocks are tiered to provide sensing 502, pre-processing504 and processing 506 for the portable device, and sensing 530,pre-processing 532 and processing 534 for the sensing device. Again, theoperating environment 500 of FIG. 5 is merely one example, and otherpossible configurations are contemplated in the present disclosure.

Starting with portable device sensing block 502, ambient sound waves arecaptured in microphone 508, multi-axis accelerometer values are capturedin accelerometer 510, and GPS coordinates are captured in GPS 512. Theraw audio data from microphone 508 is transmitted to pre-processingblock 504, in which the raw audio data is transformed into frequencycomponents, described in greater detail below. Raw GPS data from GPS 512may be transmitted to GPS pre-processing 516. In some illustrativeembodiments, the GPS pre-processing block 516 may comprise a heuristicand/or temporal preprocessing on the GPS coordinates. Locations may bederived using a density-based clustering algorithm (DBSCAN). In someillustrative embodiments, GPS traces may be anonymized by shifting theabsolute positions such that the centroid of the locations becomes thezero point of the coordinate system.

Turning to processing block 506, the illustrated sub-blocks may beconfigured for processing on a portable device (e.g., 112, 202) and/orprocessing device (e.g., 114). In this example, feature block 518receives the transformed audio data from block 514, and performs featureextraction 518 on the audio to identify characteristics of the audio. Insome illustrative embodiments, the audio feature extraction may be basedon a Fast Fourier Transorm (FFT) algorithm that samples an audio signalover a period of time or space and divides it into its frequencycomponents comprising single sinusoidal oscillations at distinctfrequencies each with their own amplitude and phase. Dominantfrequencies may be determined by measuring the divided signal over apredetermined time period. The FFT algorithm may be configured tocalculate the discrete Fourier transform (DFT) of a sequence, or itsinverse (IFFT). Fourier analysis may then be used to convert the signalfrom its original domain to a representation in the frequency domain,and vice versa. The FFT rapidly computes such transformations byfactorizing the DFT matrix into a product of sparse (mostly zero)factors. As a result, it manages to reduce the complexity of computingthe DFT from O(n²), which arises if one simply applies the definition ofDFT, to O((n)logn), where n is the data size.

In some illustrative embodiments, audio feature extraction 518 may bebased on a mel-frequency cepstrum (MFC), which represents the short-termpower spectrum of a sound received from microphone 322, based on alinear cosine transform of a log power spectrum on a nonlinear mel scaleof frequency. The MFC may be based on collective mel-frequency cepstralcoefficients (MFCCs) derived from a type of cepstral representation ofthe audio. The difference between the cepstrum and the mel-frequencycepstrum is that in the MFC, the frequency bands are equally spaced onthe mel scale, which computationally approximates the human auditorysystem's response more closely than the linearly-spaced frequency bandsused in the normal cepstrum. This frequency warping can allow for betterrepresentation of sound, for example, on processing human speech. MFCCsmay be derived by taking the Fourier transform of (a windowed excerptof) a signal, where the powers of the spectrum obtained are mapped ontothe mel scale, using triangular overlapping windows. The logs of thepowers are taken at each of the mel frequencies, and the discrete cosinetransform is taken of the list of mel log powers, as if it were asignal. The MFCCs would be the amplitudes of the resulting spectrum. Ofcourse, those skilled in the art will appreciate that there may bevariations on this process, such as using differences in the shape orspacing of the windows used to map the scale, or the addition ofdynamics features such as “delta” and “delta-delta” (first- andsecond-order frame-to-frame difference) coefficients.

In some illustrative embodiments, it has been found that anon-stationary harmonic chirp model is particularly effective atmodeling non-stationary signals, such as those likely to be produced byan ASD patient at the onset of an episode. In an example, a parametrictime-varying model may be used to capture the linear evolution of thefrequency of sinusoidal components. Here, characteristics such as thecomplex slope, introduced to capture variations of the harmoniccomponents, can be decomposed into two terms, one for frequencyadjustment and the other for the amplitude slope. This model may furtherbe extended by a second order complex polynomial for each harmoniccomponent. Accordingly, an overall estimation procedure may beperformed, based on the minimization of a Least Square criterion, wheremodel parameters may be refined using iteration in order to estimate thedifferent sinusoidal components. Such a configuration may effectivelytrack the linear evolution of the frequency of each sinusoidal componentindependently. Furthermore, the model can account for errors in theinitial estimates of the sinusoidal component frequencies.

Turning to block 520, the accelerometer processing may implement anArtificial Neural Network (ANN) that receives as input the raw dataacquired from the accelerometer, and extracts features from theaccelerometer data, including, but not limited to, mean along z-axis,maximum, minimum, standard deviation and Root Mean Square (RMS) from themagnitude of the acceleration, average of peak frequency (APF), standarddeviation, RMS, maximum and minimum along x-axis, y-axis and z-axis, andcorrelation between z-axis and y-axis. Features extracted from themagnitude of the acceleration include, but are not limited to, mean,standard deviation, cross-axis signals correlation, Fast FourierTransform (FFT) spectral energy, frequency domain entropy and log ofFFT, and implements the Naïve Bayes, C4.5 Decision Tree, K-NearestNeighbor (KNN), and SVM methods. Location features block 522 performsthe aforementioned heuristic and/or temporal preprocessing on the GPScoordinates and may derive location using DBSCAN or other suitablealgorithm.

Turning to sensing block 530, a heart rate monitor sensor 536 receivesthe raw heart rate data, which is forwarded to pre-processing block 532,where filer 538 filters the raw data and provides the filtered data toprocessing block 534 that comprises a plurality of feature extractingblocks. Time domain features block 540 may include features including,but not limited to, mean value of variation of the beat-to-beat (RR)intervals (mean RR), standard deviation of RR intervals (SDNN),root-mean-square successive difference of RR intervals (RMSSD), numberof successive difference of RR intervals which differ by more than 50 msexpressed as a percentage of total RR intervals (pNN50), and geometricmeasures, including the total number of RR intervals divided by theheight of the histogram of all RR intervals measured on a scale withbins of 1/128s (HRV index) and the triangular interpolation of RRinterval histogram (TINN).

Nonlinear Features Approximate entropy (ApEn) may be utilized in block542 to measure the complexity or irregularity of the signal, where largevalues of ApEn indicate high irregularity, and smaller values of ApEnindicate a more regular signal. The fast RR variability in the HRV data(SD1) may also be determined along with the long-term variability (SD2),where SD1 and SD2 may be considered as coefficients of the Poincoirplot. SD1/SD2 may be processed as the ratio of short interval variationto the long interval variation.

Frequency Domain Features may be processed in block 544 by estimatingthe power spectral density of the RR intervals using the Lomb-Scargleperiodogram since this algorithm is particularly advantageous with timeseries that are not necessarily evenly spaced. The spectrum may bedivided into a plurality of frequency bands, such as very low frequency(VLV), 0.01-0.04 Hz; low frequency (LF), 0.04-0.15 Hz; and highfrequency (HF), 0.15-0.4 Hz. In some illustrative embodiments, thefeatures used are the normalized values of LF, HF, and the ratio of LFand HF (LF/HF). The ratio LF/HF is not only useful as a feature fordetecting stress, but is also very important to differentiate betweensleep stages.

The outputs of processing block 506 may be combined to form a globalsensor output S1 546, while the output of processing block 534 may becombined to form another global sensor output S2. The global sensoroutputs S1, s2 may further be used to formulate a total sensor outputvalue S that is configured as a function of outputs S1 and S2. Theglobal sensor outputs S1, S2 may be configured as string value that mayor may not include partitions for each respective data portion (e.g.,518-522, 540-544), however the total and global sensor outputs shouldpreferably include markers or flags that allow the system to processindividual portions of each total and/or global sensor output asrequired.

The total sensor output S 550 may then be used as an input to learningmodule 552, classification module 554 and labeling module 556 in orderto classify and label various sensor outputs and sensor outputcombinations. Accordingly, when certain sensor conditions occur andsubsequently change, the blocks 552-556 may adapt to those changes andprovide new and/or updated context to the sensor outputs. In someillustrative embodiments, the learning module 552 may include a userinput 558 (such as 315, 317 of FIG. 3) that allows a user to manuallyprovide an input to the device and/or system. This configuration may beadvantageous for obtaining manual confirmation from the user when asensor output condition is detected. In one example, after a sensoroutput condition is detected, a processor (e.g., CPU 301) may displaythe condition to a user, along with a request for the user to verify thesensor condition. Once the sensor condition is verified or rejected, thelearning module 552 may update its settings accordingly to improveaccuracy.

FIG. 6 shows a process 600 for collecting and processing heart ratedata, accelerometer data, and audio data in a portable device to detectand alert for a state of ASD under an illustrative embodiment. In block602, the device (e.g., 402) collects heart rate data utilizing any ofthe techniques disclosed herein. In decision block 604, the devicedetermines if the heart rate data exceeds a predetermined threshold(HRT₁). If it does not (“NO”), the process 600 continues to collectheart rate data in block 602. In some illustrative embodiments, thecollection of heart rate data may be continuous, or it may be periodic,over predetermined periods of time. In some illustrative embodiments,the collection of heart rate data may be triggered by the activation ofanother sensor (e.g., light sensor, accelerometer). If, in decisionblock 604, the device determines that the heart rate exceeds thepredetermined threshold, the device (e.g., 402) begins to collectaccelerometer data in block 606. In decision block 608, the device(e.g., 402) determines if the heart rate data exceeds a second threshold(HRT₂). If not (“NO”), the device (e.g., 402) continues to collectaccelerometer data in block 606. If the heart rate is determined toexceed the second threshold (HRT₂) (“YES”), the device (e.g., 402)begins collecting audio data (e.g., via microphone 322) in block 610 toextract features and determine audio characteristics. In decision block612, the device determines if the audio features meet or exceed an audiothreshold (AT₁). In some illustrative embodiment, this threshold isbased on frequency and/or amplitude characteristics of the transformedaudio. In some illustrative embodiments, the threshold may be based ontransformed audio coefficients, and/or slope and/or variations ofharmonic components. If the device determines that the audio features donot exceed the audio threshold (“NO”), the process 500 then continues tocollect audio data 610. If, however, the device determines in block 612that the audio features exceed the threshold (“YES”), the device detectsan onset ASD episode and transmits an alert in block 614.

It should be understood by those skilled in the art that the process ofFIG. 6 may be modified to deactivate certain features ifpreviously-exceeded thresholds fall back into compliance (i.e., fallbelow the threshold). For example, if the device is collecting audiodata in block 610 and the detected heart rate falls below the secondthreshold (HRT₂), the device will stop collecting audio data. Similarly,if the detected heart rate continues to fall below the first threshold(HRT₁), the device may stop collecting accelerometer data. It shouldalso be understood by those skilled in the art that the order of datacollections (606, 610) may be altered or reversed, and that audio datamay be collected in block 606, while accelerometer data may be collectedin block 610.

FIG. 7 shows a process 700 for extracting feature vectors foraccelerometer data based on heart rate conditions, and determining audiocharacteristics for detecting and alerting for a state of ASD under anillustrative embodiment. In some illustrative embodiments, there may beinstances where heart rate data needs to be considered at or in betweenmultiple heart rate thresholds. In the process 700, after accelerometerdata is collected (“A”, see 606), decision block 702 determines if theheart rate is between a plurality of heart rate thresholds (HRT₁, HRT₂).If not (“NO”), the process returns (“B”) to collecting accelerometerdata (606). If the heart rate is within a specific heart rate band(“YES”) the device (e.g., 402) extracts accelerometer feature vectors inblock 704. In decision block 706, the device determines if predeterminedaccelerometer feature characteristics are present. If not (“NO”), theprocess returns (“B”) to collecting accelerometer data 606. If thepredetermined accelerometer feature characteristics are present (“YES”),the process 500 then moves to decision block 708, where the devicedetermines if audio features are greater than a predetermined audiothreshold (AT₁) 708. If not, (“NO”) the process 500 transmits an alertin block 712. If the audio features are greater that the predeterminedaudio threshold (“YES”), then the process transmits an alert in block710.

It should be understood by those skilled in the art that the alertsdisclosed in blocks 614, 710 and/or 712 are pursuant to a devicedetecting the onset of an ASD episode. In other words, a device may beconfigured to collect and process sensor data and transform it into atangible estimation of an ASD episode. This technical resultadvantageously allows for individuals to monitor and alert ASD patients,their friends and/or relatives, as well as health care service providersthat may be associated with the ASD patient.

In some illustrative embodiments, the system may use filters so thatalerts are selectively transmitted, based on a magnitude and/orfrequency of sensor values. The filter may be advantageously used sothat the user/patient, friends, relatives and/or service providers arenot inundated with multiple alerts for a single episode. A timer mayalso be used that is set to a default value that can be changed usingthe learning module 552.

Also, using the technologies and techniques disclosed herein,conventional operations on devices monitoring ASD are improved, byallowing the device to sense, collect and process data in a tieredfashion, using co-dependent sensor values (e.g., heart rate,accelerometer, audio) to determine the timing, order and the level ofsensing, thus reducing the overall processing overhead and strain placedon a battery.

In addition to the above disclosure, in some illustrative embodiments,sensor thresholds may be made dependent on a detected location for adevice. In some illustrative embodiments, GPS coordinates may be labeledto determine location characteristics. For example, a locations may bespecific to the user (e.g., “home”, “work”, “school”, mother's house”,etc.), or may be generically specified by type (e.g., “supermarket”,“movie theater”, “restaurant”, “car wash”, etc.). In some illustrativeembodiments, the location may be tied to a specific merchant for thelocation (e.g., “Best Buy”, “McDonalds”, etc.). During operation, thesystem (e.g., 100, 200) may initially begin with a default set ofthresholds, where each threshold may be different and specific to alocation and/or location characteristic. In some illustrativeembodiments, thresholds may be the same for locations having similarlocation characteristics.

As the operating environment (e.g., 500) continues operation, themodules 552-556 may “learn” the effects these locations may have on auser and adjust sensor parameters and thresholds for any sensoraccordingly. For example, if a user's GPS location shows them to be at agym, the heart rate and accelerometer parameters/thresholds may beadjusted upwards to reflect the physical activity being performed by theuser. Because the sensor thresholds are increased, this reduces thepossibility of detecting false positives. Similarly, if a user has beenpreviously detected to have had one or more episodes at a specificlocation, the sensor thresholds may be adjusted downward in order toallow the system to detect onset ASD episodes earlier. In someillustrative embodiments, sensor thresholds may be moved up and/or downindependently of each other, depending on the application.

FIG. 8 shows a process 800 for collecting and processing globalpositioning system (GPS) data for loading thresholds for use in sensordata processing under an illustrative embodiment. In block 802 thedevice may collect GPS data. In decision block 804, the devicedetermines if the GPS data reflects a saved location. If the location issaved (“YES”), the process 800 loads thresholds in block 806 that areassociated with the saved location. If the location is not saved (“NO”),the process 800 moves to decision block 810 to see if there is alocation characteristic history. If not (“NO”), the system loads thedefault thresholds in block 812. If a location characteristic historyexists (“YES”), the system loads historical thresholds for theassociated location characteristic in block 814.

The figures and descriptions provided herein may have been simplified toillustrate aspects that are relevant for a clear understanding of theherein described devices, structures, systems, and methods, whileeliminating, for the purpose of clarity, other aspects that may be foundin typical similar devices, systems, and methods. Those of ordinaryskill may thus recognize that other elements and/or operations may bedesirable and/or necessary to implement the devices, systems, andmethods described herein. But because such elements and operations areknown in the art, and because they do not facilitate a betterunderstanding of the present disclosure, a discussion of such elementsand operations may not be provided herein. However, the presentdisclosure is deemed to inherently include all such elements,variations, and modifications to the described aspects that would beknown to those of ordinary skill in the art.

Exemplary embodiments are provided throughout so that this disclosure issufficiently thorough and fully conveys the scope of the disclosedembodiments to those who are skilled in the art. Numerous specificdetails are set forth, such as examples of specific components, devices,and methods, to provide this thorough understanding of embodiments ofthe present disclosure. Nevertheless, it will be apparent to thoseskilled in the art that specific disclosed details need not be employed,and that exemplary embodiments may be embodied in different forms. Assuch, the exemplary embodiments should not be construed to limit thescope of the disclosure. In some exemplary embodiments, well-knownprocesses, well-known device structures, and well-known technologies maynot be described in detail.

The terminology used herein is for the purpose of describing particularexemplary embodiments only and is not intended to be limiting. As usedherein, the singular forms “a”, “an” and “the” may be intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. The terms “comprises,” “comprising,” “including,” and“having,” are inclusive and therefore specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. The steps, processes, and operations described herein are notto be construed as necessarily requiring their respective performance inthe particular order discussed or illustrated, unless specificallyidentified as a preferred order of performance. It is also to beunderstood that additional or alternative steps may be employed.

When an element or layer is referred to as being “on”, “engaged to”,“connected to” or “coupled to” another element or layer, it may bedirectly on, engaged, connected or coupled to the other element orlayer, or intervening elements or layers may be present. In contrast,when an element is referred to as being “directly on,” “directly engagedto”, “directly connected to” or “directly coupled to” another element orlayer, there may be no intervening elements or layers present. Otherwords used to describe the relationship between elements should beinterpreted in a like fashion (e.g., “between” versus “directlybetween,” “adjacent” versus “directly adjacent,” etc.). As used herein,the term “and/or” includes any and all combinations of one or more ofthe associated listed items.

Although the terms first, second, third, etc. may be used herein todescribe various elements, components, regions, layers and/or sections,these elements, components, regions, layers and/or sections should notbe limited by these terms. These terms may be only used to distinguishone element, component, region, layer or section from another element,component, region, layer or section. Terms such as “first,” “second,”and other numerical terms when used herein do not imply a sequence ororder unless clearly indicated by the context. Thus, a first element,component, region, layer or section discussed below could be termed asecond element, component, region, layer or section without departingfrom the teachings of the exemplary embodiments.

The disclosed embodiments may be implemented, in some cases, inhardware, firmware, software, or any tangibly-embodied combinationthereof. The disclosed embodiments may also be implemented asinstructions carried by or stored on one or more non-transitorymachine-readable (e.g., computer-readable) storage medium, which may beread and executed by one or more processors. A machine-readable storagemedium may be embodied as any storage device, mechanism, or otherphysical structure for storing or transmitting information in a formreadable by a machine (e.g., a volatile or non-volatile memory, a mediadisc, or other media device).

In the drawings, some structural or method features may be shown inspecific arrangements and/or orderings. However, it should beappreciated that such specific arrangements and/or orderings may not berequired. Rather, in some embodiments, such features may be arranged ina different manner and/or order than shown in the illustrative figures.Additionally, the inclusion of a structural or method feature in aparticular figure is not meant to imply that such feature is required inall embodiments and, in some embodiments, may not be included or may becombined with other features.

In the foregoing Detailed Description, it can be seen that variousfeatures are grouped together in a single embodiment for the purpose ofstreamlining the disclosure. This method of disclosure is not to beinterpreted as reflecting an intention that the claimed embodimentsrequire more features than are expressly recited in each claim. Rather,as the following claims reflect, inventive subject matter lies in lessthan all features of a single disclosed embodiment. Thus the followingclaims are hereby incorporated into the Detailed Description, with eachclaim standing on its own as a separate embodiment.

What is claimed is:
 1. A portable system for tiered, co-dependent sensoractivation for monitoring onset Autism Spectrum Disorder (ASD),comprising: a processing apparatus; a memory operatively coupled to theprocessing apparatus; a communication interface, operatively coupled tothe processing apparatus; and an input/output interface, configured toreceive sensor data comprising heart rate sensor data, accelerometersensor data and audio sensor data, wherein the processing apparatus isconfigured to determine if the heart rate sensor data meets or exceeds afirst predetermined threshold, activate the collection of accelerometersensor data and extract accelerometer data features if the heart rateexceeds the first threshold, determine if the heart rate sensor datameets or exceeds a second predetermined threshold, activate thecollection of audio sensor data and extract audio data features if theheart rate sensor data meets or exceeds the second predeterminedthreshold, determine if the audio sensor data meets or exceeds apredetermined audio threshold, and transmit an alert indicating thedetection of an onset ASD episode, based on the extracted audio datafeatures, the extracted accelerometer data features and the heart ratedata.
 2. The system of claim 1, wherein the processing apparatus isfurther configured to process GPS data to determine a location of theprocessing apparatus; and set at least one of the first predeterminedthreshold and the second predetermined threshold based on the determinedlocation.
 3. The system of claim 1, wherein the heart rate datacomprises time domain features and frequency domain features.
 4. Thesystem of claim 1 wherein the processing apparatus is configured toextract accelerometer data features utilizing an artificial neuralnetwork (ANN).
 5. The system of claim 1, wherein the processingapparatus is configured to extract audio data features by transformingambient audio into frequency components.
 6. The system of claim 5,wherein the processing apparatus is configured to extract audio datafeatures utilizing a harmonic chip model applied to the frequencycomponents.
 7. The system of claim 1, wherein the input/output interfaceis configured to receive a feedback in response to transmitting thealert, and wherein the processing apparatus is configured to modify atleast one of the first predetermined threshold and second predeterminedthreshold based on the received feedback.
 8. A processor-based methodfor tiered, co-dependent sensor activation for monitoring onset AutismSpectrum Disorder (ASD) in a portable device, comprising: receiving, atan input/output interface, sensor data comprising heart rate sensordata; determining, in a processing apparatus, if the heart rate sensordata meets or exceeds a first predetermined threshold; activating, viathe processing apparatus, the collection of accelerometer sensor dataand extracting accelerometer data features if the heart rate exceeds thefirst threshold, determining, via the processing apparatus, if the heartrate sensor data meets or exceeds a second predetermined threshold,activating, via the processing apparatus, the collection of audio sensordata and extract audio data features if the heart rate sensor data meetsor exceeds the second predetermined threshold, determining, via theprocessing apparatus, if the audio sensor data meets or exceeds apredetermined audio threshold, and transmitting, via a communicationsinterface coupled to the processing apparatus, an alert indicating thedetection of an onset ASD episode, based on the extracted audio datafeatures, the extracted accelerometer data features and the heart ratedata.
 9. The method of claim 8, further comprising: process, via theprocessing apparatus, GPS data to determine a location of the device;and setting, via the processing apparatus, at least one of the firstpredetermined threshold and the second predetermined threshold based onthe determined location.
 10. The method of claim 8, wherein the heartrate data comprises time domain features and frequency domain features.11. The method of claim 8 wherein extracting the accelerometer datafeatures comprises utilizing an artificial neural network (ANN).
 12. Themethod of claim 8, wherein extracting audio data features comprisestransforming ambient audio into frequency components.
 13. The method ofclaim 12, wherein extracting audio data features comprises utilizing aharmonic chip model applied to the frequency components.
 14. The methodof claim 8, further comprising: receiving, via an input-outputinterface, a feedback in response to transmitting the alert; andmodifying, via the processing apparatus, at least one of the firstpredetermined threshold and second predetermined threshold based on thereceived feedback.
 15. A portable system for tiered, co-dependent sensoractivation for monitoring onset Autism Spectrum Disorder (ASD),comprising: a processing apparatus; a memory operatively coupled to theprocessing apparatus; a communication interface, operatively coupled tothe processing apparatus; and an input/output interface, configured toreceive sensor data comprising heart rate sensor data, accelerometersensor data and audio sensor data, wherein the processing apparatus isconfigured to activate the collection of accelerometer sensor data andextract accelerometer data features comprising frequency components ifthe heart rate exceeds a first threshold, activate the collection ofaudio sensor data and extract audio data features comprising frequencycomponents if the heart rate sensor data meets or exceeds a secondpredetermined threshold, determine if the audio sensor data meets orexceeds a predetermined audio threshold comprising frequencycharacteristics, and transmit an alert indicating the detection of anonset ASD episode, based on the extracted audio data features, theextracted accelerometer data features and the heart rate data.
 16. Thesystem of claim 15, wherein the processing apparatus is furtherconfigured to process GPS data to determine a location of the processingapparatus; and set at least one of the first predetermined threshold andthe second predetermined threshold based on the determined location. 17.The system of claim 15, wherein the heart rate data comprises timedomain features and frequency domain features.
 18. The system of claim15, wherein the processing apparatus is configured to extractaccelerometer data features utilizing an artificial neural network(ANN).
 19. The system of claim 15, wherein the processing apparatus isconfigured to extract audio data features by transforming ambient audiointo frequency components.
 20. The system of claim 15, wherein theinput/output interface is configured to receive a feedback in responseto transmitting the alert, and wherein the processing apparatus isconfigured to modify at least one of the first predetermined thresholdand second predetermined threshold based on the received feedback.